I was quite pleased with this discovery. Previously the initial params had to be subsetted to be the correct length for the conditioned model. That's not only a faff, but also I get a bit scared whenever there's direct VarInfo manipulation like this.

Now, if you use InitFromParams with a NamedTuple/Dict that has extra params, the extra params are just ignored. So no need to subset it at all, just pass it through directly!

unfortunately set_namedtuple! is used elsewhere in this file (though it won't appear in this diff) so we can't delete it (yet)

The behaviour of SampleFromPrior used to be: if the key is present, don't actually sample, and if it was absent, sample. This if/else replicates the old behaviour.

For pMCMC, this Boolean field essentially replaces the del flag. Instead of set_all_del and unset_all_del we construct new TracedModel with this set to true and false respectively.

Case in point:

julia> using Turing, StableRNGs

julia> dist = GeneralizedExtremeValue(0, 1, 0.5); @model m() = x ~ dist
m (generic function with 2 methods)

julia> mean(dist)
1.5449077018110322

julia> mean(sample(StableRNG(468), m(), HMC(0.05, 20), 10000; progress=false))
Mean
  parameters      mean
      Symbol   Float64

           x    3.9024


julia> mean(sample(StableRNG(469), m(), HMC(0.05, 20), 10000; progress=false))
Mean
  parameters      mean
      Symbol   Float64

           x    1.5868

Why is this needed? Doesn't MH just use init to get initial values like any other sampler, and then evaluate logpdfs on proposed steps?

Also, this looks like dynamical dispatch, since the context type depends on haskey(vi, vn), which could be a performance issue.

The rationale is in the comment right below this:

    # Allow MH to sample new variables from the prior if it's not already present in the
    # VarInfo.

This preserves the old behaviour of MH (see

It directly mimics the old code for SampleFromPrior: https://github.com/TuringLang/DynamicPPL.jl/blob/c1d9b61933a6251fbeac249e8ebf03631f1f25bf/src/context_implementations.jl#L131-L171 which includes the haskey(vi, vn) check.

dynamic dispatch

I don't really understand what is so bad with this. (For what it's worth, I searched online and I can't find any actual explanation, so feel free to enlighten me.) I am just calling a different method, depending on the value of a Boolean which — sure — you can't know it for sure until runtime.

But if the 'not known until runtime' is the main problem, then it seems like literally any code that uses if haskey(vi, vn) is already problematic — would it be any better or worse if I were to inline the definitions here?

if haskey(vi, vn)
    # inline the definition of tilde_assume on DefaultContext here
else
    # inline the definition of tilde_assume on Init here
end

Surely that's okay, right, because otherwise it seems incredibly limiting for programming if you can't even use conditionals. So I assume I'm missing something. Like, what's the difference between 'dynamic dispatch' and 'code that uses conditionals because values aren't known until runtime'.

I'm not sure exactly how bad dynamic dispatch is either, and I wondered about the same question you are asking. Clearly some of it has to be okay. One difference to inlining is that without inlining the right method will have to be looked up from some method table at runtime. I'm not sure what exactly that causes, but maybe there are substantial costs there? Maybe those method tables are quite heavy data structures, compared to what the code itself is dealing with?

One the first point, I don't understand why MH needs to do this differently from every other sampler. I feel like whether to fall back to InitContext when evaluating logp for sampling purposes and hitting a tilde_assume!! for an unknown variable should be a decision made on a higher level than a sampler's implementation. I get that this is not new behaviour in this PR, though it's clunkyness is exposed in a different way because we now need to create a whole new context for MH.

dynamic dispatch

I searched a bit more and came across this: https://discourse.julialang.org/t/curious-about-the-internals-of-dynamic-dispatch/102888/8. In this example, there is some object a::A but A is an abstract type, so the compiler can't figure out which method f(a) refers to ahead of time and this is only resolved at runtime.

But here the compiler can perfectly figure out which methods are being used, because both InitContext and DefaultContext are concrete types. My impression is that what's going on here is not dynamic dispatch — the dispatch itself is static: when I write tilde_assume(...) the compiler has enough information to know which method is being referred to. Once haskey(vi, vn) is evaluated, there is no more looking things up in method tables. I think the dynamic dispatch thing only happens if the types of the arguments to tilde_assume!! are themselves not concrete.

The only downside from the compiler's point of view is that it can't aggressively cut out one branch ahead of time, because it can't figure out which branch it will go down.

Also, there would be issues with type stability if both tilde_assume!!s return different types. But that's not a dispatch problem, that's just plain old type stability and that wouldn't be affected if the function definitions were inlined.

I don't understand why MH needs to do this differently from every other sampler

I don't know either. I am in principle not opposed to cutting this out and just using DefaultContext, but I would probably prefer to not make this decision in this PR.

performance

If it's of any reassurance, I ran the following code on main plus this PR; main comes in at around 0.89 seconds, this PR at 0.83 seconds. (of course, I have no clue why or what made it better! 😄)

using Turing

J = 8
y = [28, 8, -3, 7, -1, 1, 18, 12]
sigma = [15, 10, 16, 11, 9, 11, 10, 18]

@model function eight_schools_centered(J, y, sigma)
    mu ~ Normal(0, 5)
    tau ~ truncated(Cauchy(0, 5); lower=0)
    theta = Vector{Float64}(undef, J)
    for i in 1:J
        theta[i] ~ Normal(mu, tau)
        y[i] ~ Normal(theta[i], sigma[i])
    end
end

model_esc = eight_schools_centered(J, y, sigma)
@time sample(model_esc, MH(), 10000; progress=false);

Like with MH, I wonder about dynamic dispatch here. Might be unavoidable and/or inconsequential in this case though.

I don't see why these became necessary now. Is it something about the type hierarchy around Sampler?

Yeah, RepeatSampler is weird, because it doesn't subtype InferenceAlgorithm. So some of the code like default sample methods has to be copied over for it. In the past, the methods used to be written for Union{Sampler{<:InferenceAlgorithm},RepeatSampler}. I think that's worse because it was less flexible, you couldn't change one without affecting the behaviour of the other.

Leaving this comment just as a reminder that this needs to be removed before merging.

Likewise this.

Likewise a reminder comment.

Leaving a comment to track that this gets done before merging.

I'm not sure exactly how bad dynamic dispatch is either, and I wondered about the same question you are asking. Clearly some of it has to be okay. One difference to inlining is that without inlining the right method will have to be looked up from some method table at runtime. I'm not sure what exactly that causes, but maybe there are substantial costs there? Maybe those method tables are quite heavy data structures, compared to what the code itself is dealing with?

One the first point, I don't understand why MH needs to do this differently from every other sampler. I feel like whether to fall back to InitContext when evaluating logp for sampling purposes and hitting a tilde_assume!! for an unknown variable should be a decision made on a higher level than a sampler's implementation. I get that this is not new behaviour in this PR, though it's clunkyness is exposed in a different way because we now need to create a whole new context for MH.